Papers with failure recovery
One Battle After Another: Probing LLMs’ Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for instruction-following in multi-topic dialogues are limited to a fixed number of turns, susceptible to saturation and failing to account for users’ interactive experience. |
| Approach: | They propose a framework featuring a three-layer tracking mechanism and a query synthesis agent to mimic sequential user behaviors. |
| Outcome: | The proposed framework outperforms existing benchmarks in the evaluation of instruction following in multi-topic dialogues and demonstrates deficiencies in failure recovery and fine-grained instruction following. |
Tools Fail: Detecting Silent Errors in Faulty Tools (2024.emnlp-main)
Copied to clipboard
| Challenge: | a failure in one tool can trigger a cascade of errors, leading to complete task failure. |
| Approach: | They propose a framework for tools more broadly which explores a model’s ability to detect “silent” tool errors and reflect on how to plan. |
| Outcome: | The proposed approach shows that the model can detect "silent" tool errors and plan. |